Chapter for MARKOV DECISION PROCESSES

Accelerated decomposition techniques for large discounted Markov decision processes

Many hierarchical techniques to solve large Markov decision processes (MDPs) are based on the partition of the state space into strongly connected components (SCCs) that can be classified into some levels. In each level, smaller problems named restricted MDPs are solved, and then these partial solutions are combined to obtain the global solution. In this paper, we first propose a novel algorith...

Bayesian approaches to Learning and Decision Mak- ing

Behavioural phenomena are central to psychiatric disorders. Computational modelling allows the learning and decision-making processes underlying behaviour to be modelled in great detail. By doing so, specific and possibly highly complex hypotheses about the underlying processes can be directly tested on the data. The first part of this chapter introduces Markov Decision Problems (MDPs) as a for...

From Markov Chains to Stochastic Games

Markov chains1 and Markov decision processes (MDPs) are special cases of stochastic games. Markov chains describe the dynamics of the states of a stochastic game where each player has a single action in each state. Similarly, the dynamics of the states of a stochastic game form a Markov chain whenever the players’ strategies are stationary. Markov decision processes are stochastic games with a ...

Decision making with inference and learning methods

In this work we consider probabilistic approaches to sequential decision making. The ultimate goal is to provide methods by which decision making problems can be attacked by approaches and algorithms originally built for probabilistic inference. This in turn allows us to directly apply a wide variety of popular, practical algorithms to these tasks. In Chapter 1 we provide an overview of the gen...

Chapter 13 CONSTRAINED DISCOUNTED SEMI - MARKOV DECISION PROCESSES

Screening and Treatment of Chronic Diseases

In recent years, Markov decision processes (MDPs) and partially observable Markov decision processes (POMDPs) have found important applications to medical decision making in the context of prevention, screening, and treatment of diseases. In this chapter, we provide a review of state-of-the-art models and methods that have been applied to chronic diseases. We provide a tutorial about how to for...